Theron Pummer's impressive new book The Rules of Rescue seeks to defend effective altruism without taking on the controversial moral theoretical commitments. Through an exploration of the framework of requiring reasons and permitting reasons that is the backbone of his argument, this article raises some doubts about how successful Pummer's strategy of avoidance can be.

The creation of artificial moral systems requires us to make difficult choices about which of varying human value sets should be instantiated. The industry-standard approach is to seek and encode moral consensus. Here we argue, based on evidence from empirical psychology, that encoding current moral consensus risks reinforcing current norms, and thus inhibiting moral progress. However, so do efforts to encode progressive norms. Machine ethics is thus caught between a rock and a hard place. The problem is particularly acute when (...) progress beyond prevailing moral norms is particularly urgent, as is currently the case due to the inadequacy of prevailing moral norms in the face of the climate and ecological crisis. (shrink)